1. Field of the Invention
The present invention relates to an image processing apparatus, an image sensor, a video signal processing method and a computer program. More specifically, the present invention relates to a video signal processing apparatus and an image sensor, each including a color filter composed of a combination of long-pass filters, and a video signal processing method and a computer program.
2. Description of the Related Art
Monitoring cameras and commercial video cameras typically capture visible-light images under visible light condition of natural light during daytime. During nighttime, the cameras typically capture infrared-light images of infrared light reflected from a subject in response to an infrared light beam projected by an infrared light projector. When image capturing is performed under natural light, an infrared-light cut filter is mounted on the camera. When image capturing is performed using infrared light, the infrared-light cut filter is removed. Such a camera is disclosed in Japanese Unexamined Patent Application Publication No. 2002-084451. If the infrared-light cut filter is mechanically removed, a moving portion is attached to mount and demount the filter on an optical system of the cameral, leading to a cost increase of the camera.
In such a filter switching system, the camera operates to capture the infrared-light image or the visible-light image. The camera cannot capture both the infrared-light image and the visible-light image at the same time.
Japanese Unexamined Patent Application Publication No. 2004-056343 discloses a system that performs data communications by means of infrared light. The system receives blinking infrared light responsive to data communications and analyses the received infrared light. The system includes an image sensor to receive the infrared light. Such a system, if provided with a removably attached mechanism for infrared light, cannot receive both the infrared light and picture data in visible light at the same time.
Widely available color filter arrays are illustrated in FIGS. 1A and 1B. FIG. 1A illustrates a color filter array of red (R), green (G) and blue (B). FIG. 1B illustrates a combination of white (Y) as a luminance signal and red (R), green (G) and blue (B). If an attempt is made to capture images with an image sensor having such an array and without infrared-light cut filter under an infrared-light abundant light source, a large amount of infrared light is captured. Color reproduction and image quality are degraded.
Japanese Unexamined Patent Application Publication No. 11-144031 discloses a system that allows each of an infrared light image and a visible light image to be captured. With this system, background is identified using infrared light. A plurality of image sensors are incorporated to capture each of the infrared light image and visible light image in the system, leading to a cost increase.
Concurrent retrieval of infrared-light images and color images typically excellent in color reproduction property are useful in removal of scratches on a film, skin color detection and noise reduction. There is a mounting need for a mechanism that captures both infrared images and color images and processes both the infrared image and the color images.
In one contemplated mechanism for capturing and processing both the infrared image and the color image, an infrared-light component is removed through an image process by adding an infrared-light transmission pixel to a filter of existing color filter array as shown in FIG. 1. However, infrared light of all color components cannot be removed to a sufficient level even if a signal resulting from a pixel derived from an infrared-light transmissive filter is used. This is because filter characteristics of a filter used as a color filter array is not sufficiently close to perfect ones.
The color filter array of pixels in an image sensor is typically a combination of at least one of a plurality of bandpass color filters of blue, green, cyan and magenta, and a long-pass color filter of red or yellow. Each bandpass filter permits light in a predetermined wavelength region (band). Each long-pass filter permits light above a predetermined wavelength. The color filter discussed with reference to FIG. 1 is composed of such bandpass filters and long-pass filter. Ideally, each bandpass filter permits only light falling within a predetermined wavelength range needed for color reproduction to pass therethrough while blocking the remaining light having a wavelength longer and shorter than the predetermined wavelength range.
Most of materials of the bandpass filters on the color filter array function as a bandpass filter only in a visible-light wavelength range. Most of the materials of the bandpass filter permit partially the infrared light to pass therethrough, thereby failing to function as a bandpass filter within the infrared light range as invisible light. Transmission characteristics of the infrared light passing through the bandpass filter change depending on the type of each bandpass filter.
In one related art, the infrared-light component is removed based on a signal captured using a bandpass filter. In this technique, received infrared light is calculated with an infrared light transmission pixel added to a color filter array. An amount of infrared light having passed through each bandpass filter is estimated, and the infrared-light component is removed based on the estimation results.
The filter used to obtain the infrared-light transmission pixel is a long-pass filter and transmissive characteristics of the filter in infrared light in a pass band remain constant regardless of wavelength. On the other hand, infrared light transmissive characteristics of each bandpass filter that permits one of blue, green, cyan, magenta light to pass selectively are different depending on each bandpass filter. It is thus difficult to remove accurately only the infrared-light component from the bandpass filter different in filter characteristics using the signal alone acquired from the infrared light transmission pixel. Infrared light removal performance required to capture a color image having excellent color reproduction property cannot be obtained.